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Validating the National Survey of Student Engagement Against 
Student Outcomes: Are They Related? 



While there exist many examples of institutional use of the results of the National Survey of 
Student Engagement (NSSE), there is a relative paucity of research explicitly linking student 
outcomes to responses on the survey. A major Doctoral-Extensive institution in the Southeast 
recently conducted a large-scale implementation of the National Survey of Student Engagement 
(NSSE). We have linked multiple years of NSSE responses to several student outcomes: 
freshman retention, GPA, pursuit of graduate education, and employment outcome upon 
commencement/degree conferral. Our research finds minimal explanatory power in the NSSE 
benchmarks for these outcomes. A statistically-derived model from the individual NSSE items 
shows greater promise, although there are difficulties in replicating the model for previous 
student cohorts. 



Introduction 

A large body of research on student learning has concluded that students who are actively 
involved in academic and co-curricular activities gain more from their college experience than 
students who are not as involved (Pascarella and Terenzini, 1991, 2005; Astin, 1977, 1993). 
Research on student engagement has engendered a national debate among higher education 
scholars and practitioners. Chickering and Gamson (1987) most famously described seven 
principles of good practice in undergraduate education, noting that effective educational practice 
includes: 

• student-faculty contact 

• cooperation among students 

• active learning 

• receiving prompt feedback 

• student time on task 

• communication of high expectations 

• respect for diverse talents and ways of learning 

In addition to these practices, effective educational outcomes are associated with an institutional 
environment that is . .perceived by students as inclusive and affirming and where expectations 
for performance are clearly communicated and set at reasonably high levels” (NSSE, 2003). 

The National Survey of Student Engagement (NSSE) has developed an instrument that is 
designed to measure student engagement and the degree to which institutions provide students 
with an effective learning environment (Kuh, et al. 2001; Kuh 2001a). The survey uses a well- 
developed, validated set of items directed at a variety of student behaviors and experiences 
related to engagement. A major component of the way NSSE results are reported is through its 
benchmark scales. These scales are informed partially by an empirically derived grouping of 
survey items as well as an intuitive understanding of concepts proposed by Astin’s (1984) theory 
of student involvement, and by Chickering & Gamson’s (1987) work (Kuh, et al. 2001). 

The five NSSE benchmarks — Level of Academic Challenge, Active and Collaborative Learning, 
Student-Faculty Interaction, Enriching Educational Experiences, and Supportive Campus 
Environment — serve as the framework around which the NSSE annual reports are organized. The 
benchmarks, essentially unweighted indices of items, are intended to be a useful tool for internal 
evaluation, and are also used to facilitate comparisons among other institutions and institutional 
types (Kuh, 2001). 



The benchmarks, however, are not the only way to look at NSSE data. Alternative, empirically 
derived scales have been developed from the data. For example, Zhao and Kuh (2004) created 
scales to measure perceived gains in social, practical, and academic competence. Similarly, Pike 
(2004a), based on Wainer and Kiely’s (1987) testlet concept, developed a set of “scalelets” — an 
alternative grouping of items that further disaggregates the NSSE benchmarks’ conceptual 
domains. The scalelets were developed as a way to provide actionable unit and institution-level 
information on engagement. 

In addressing NSSE data, institutions need to be mindful of the desired goals they have for their 
students and themselves. The goal of an institution should not be to achieve higher NSSE 
benchmark scores for the sake of doing so, but rather to gain keener insight into the relationship 
between student engagement (as measured by NSSE) and the desired outcomes the institution has 
for its students. “Success” has different meanings for different institutions, and the means by 
which success may be measured will vary as well. For example, academic success can be defined 
by persistence and graduation. Success can also be defined by whether or not a graduate is able to 
pursue a meaningful career as a result of their education or if the graduate reports satisfaction 
with their college experience. Success for some students may not even include graduation, but 
rather the attainment of marketable skills (Pfeiffer, 1998). 

Institutions have a variety of means to actually measure the degree to which its students achieve 
these goals. For example, academic success might be measured by grades or graduation rates. 

Job satisfaction can be gauged by responses to alumni surveys. Student satisfaction can be 
assessed by retention rates, student self-reports, or even contributions to the institution’s alumni 
association (Pascarella and Terenzini, 2005). 

Given the demonstrated connection as reported in the literature between engagement and positive 
student experience, and the intended design of NSSE in measuring this engagement, one would 
expect to find associations between NSSE results and measures of desired outcomes. While there 
are many examples of institutions using NSSE to explore the impact of programs that arc 
intended to affect student engagement, there are at present relatively few studies on the 
relationship between measures of engagement and measures of student outcomes. 1 

Several recent studies have utilized engagement measures similar - to the NSSE benchmarks to 
evaluate self-reported gains in learning outcomes. The earliest of these (Kuh, Pace, and Vesper 
1997) is really best described as a research precursor to the NSSE — developing instrumentation 
to link behaviors to engagement and learning outcomes. Items were taken from the College 
Student Experiences Questionnaire (Pace, 1990) to derive a set of “best practices” scales (faculty- 
student contact, cooperation among students, and active learning), and a set of “process 
indicators” for learning (general education and intellectual skills). The research found a very clear 
connection between the “best practices” and “process indicators.” One important area that was 
not addressed in that study was external validation — comparing these measures against outcome 
measures external to this single instrument. 

A limited foray into combining sources of information internal and external to the NSSE, was a 
feature of Zhao and Kuh’s (2004) study on the impact of learning communities on engagement 
and academic outcomes. That NSSE-era study furthermore utilized an alternative (non- 
benchmark) set of measures based on the NSSE survey data — including three learning outcome 
measures (personal and social development, general education, and practical competence) drawn 



For examples of how NSSE is used by institutions, see http://webdb.iu. edu/Nsse/?view=tools/using_nsse_data. 
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from some of the non-benchmark items on the survey. This research demonstrated that 
participation in learning communities had an impact on measures of student engagement, as well 
as between learning community participation and student outcomes (both GPA and self-reported 
gains). However, these effects were dependent on controlling for initial student differences — the 
background characteristics of community participants and non-participants had differential 
impact on student engagement. Zhao and Kuh's study demonstrated that individual behavioral (or 
intentional) items can be used effectively in predicting outcomes, even across multiple (n=365) 
institutions with a variety of different programs. It also demonstrated the functional use of non- 
benchmark measures in evaluating engagement. However, direct connections between broader 
engagement measures and student outcomes were not investigated. 

Kuh, Pace, and Vesper (1997) briefly addressed the differential impact of engagement between 
institutions. This subject has been explored in many studies, almost all of which show the 
majority of any differences disappeared when student characteristics were controlled (Pascarella 
and Terenzini, 1991). Pike, Kuh, and Gonyea (2003) explored a specific and relatively untested 
effect that institutional mission — proxied by Carnegie classification — may have on engagement. 
As with much of the prior research, differences were found, but these could be attributed to 
differences in the student populations among these types of institutions, with no statistical 
interaction between student and mission. Clearly institutional differences in engagement are 
impacted in large paid by the student body an institution attracts. This may be a product of 
institution type, but it is also somewhat of a character or reputation issue as well. 

A recent study by Carini, Kuh, & Klein (2006) includes one of the most extensive evaluations of 
the associations of NSSE benchmarks and student outcomes. That research was conducted over 
14 institutions of varying characteristics. In evaluating learning outcomes, the authors chose to 
use more direct measures of academic success (i.e. GPA) and standardized testing to measure 
ability. The results suggested that there was a link between engagement measures and educational 
outcomes. However, these statistical associations were quite small and rarely met cutoffs for even 
small effect sizes. For the whole population, no individual NSSE benchmarks accounted for more 
than 2% of the variance in their outcome measures. In fact, the best measures came from other 
NSSE engagement and learning scales. Important to note in this study is that when the students 
were divided by pre-college ability (as measured by SAT scores), more powerful associations 
were found between performance and engagement for the lowest ability students. Differences 
between institutions were also noted not just in engagement, but in the connection between 
engagement and educational outcomes — some producing better than or worse than expected 
outcomes (see also Kuh, Carini, & Klein, 2004). The authors termed this relationship between 
engagement and educational outcomes “educational efficiency.” This presents an interesting 
question regarding what is occurring for those institutions with low engagement-outcomes 
associations to produce “better-than-expected” outcomes. The research also indicates a caveat in 
using benchmarks: measures of engagement may not associate strongly with outcomes for every 
institution. 



2005 [author’s institution] / NSSE Benchmarks - Analysis 

For the 2005 administration of NSSE, [author’s institution] was able to take advantage of funds 
provided by the University System of [xxx] to sample our entire first-year and senior populations. 
With a total response rate of 47.3 percent, we obtained data from 1,244 first-year students and 
629 seniors. This robust sample formed the basis for our subsequent exploratory efforts. 
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In our attempts to link NSSE results with student outcomes, we were limited to what we could 
plausibly measure in our student population. We do not yet have the ability to conduct a long- 
term follow-up of students who have taken the NSSE. However, we do routinely conduct 
baccalaureate and graduate alumni surveys, and hope to include these results in the future. We 
chose to focus on four indicators of success in our student body: cumulative GPA, first-year 
retention, job attainment upon graduation, and the decision to pursue a graduate degree. 
Operationally, we examined the cumulative GPA in the spring semester in which the students 
completed the NSSE (both first-years and seniors). For retention, a student was considered 
retained if they returned to [author’s institution] for the Fall 2005 semester — we did not look at 
retention/graduation in the senior respondents, as virtually all seniors either graduated or returned 
for an additional term. For job attainment and post-baccalaureate education plans, we examined 
student responses to the [author’s institution] Commencement Survey, which is given to students 
on the day of commencement ceremonies. We recognize the limitations of this measure — many 
students who do not report employment at graduation are often gainfully employed six months 
later. Still, since 64% of [author’s institution] graduates report employment upon graduation or 
placement in a graduate program, the data may be deemed a relatively good indicator of eventual 
success for [author’s institution] graduates. 

We were interested in examining the following research questions: 

1) to what degree are NSSE benchmarks correlated with the student outcomes described 
above; 

2) to what degree are the NSSE scalelets developed by Pike (2004) correlated with student 
outcomes and; 

3) can a model be generated that provides a better fit to student data at [author’s institution]? 
In other words, what NSSE items are most associated with positive student outcomes at 
the Institute? 

Due to the technological focus and high concentration of Engineering programs that characterize 
[author’s institution], we do not expect that the models generated in this paper will necessarily 
work at other institutions. However, our hope is that other institutions can use these methods to 
investigate the relationships between NSSE and their student bodies in a similar manner. 

The manner in which our analyses proceeded is as follows: first, we examined some of the 
psychometric properties of the NSSE benchmarks and explored the correlations between the 
benchmarks and student outcomes. We then looked at the alternative groupings of NSSE items 
generated by Pike’s (2004) work. Next, we used regression analysis (OLS) in an attempt to 
improve the predictive power of NSSE variables on student outcomes. Our approach here was 
data driven: we wished to see if there arc any NSSE items not presently included in the 
benchmarks and scalelets that contributed to desired student outcomes. Forward regression was 
used to generate a “best fit” model for each outcome. Finally, we tested the model on the results 
of the 2003 NSSE to see if its predictive power held up on a different student cohort. 

Reliability of the NSSE Benchmarks 

The benchmarks arc a blend of theory and empirical analysis, and while they have been adopted 
by NSSE, the consistency of some of the scales may be an object of concern. Generally when 
interpreting scales, one wishes to achieve a high level of internal consistency between each of the 
items in the scale. In other words, one expects that respondents answer similar questions in 
similar ways. A conventional measure of internal consistency is Cronbach’s alpha, a measure of 
the inter-correlation of the items that make up a scale. An alpha score of .70 is considered 
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minimally adequate in creating scales. 2 Cronbach’s alphas were calculated for the [author’s 
institution] 2005 survey respondents and compared to those generated by NSSE in 2002 to create 
the original benchmark scales. The results are presented below; bolded values are those that fall 
below the .70 threshold. 



Table 1. Cronbach’s Alpha Values for Benchmark Scales: National NSSE and [author’s institution] 
Results 



[author’s institution] 
NSSE 2002 2005 



Benchmark 


First-Year 


Senior 


First-Year 


Senior 


Level of Academic Challenge 


0.74 


0.76 


0.66 


0.68 


Active and Collaborative Learning 


0.64 


0.65 


0.58 


0.58 


Student-Faculty Interaction 


0.72 


0.75 


0.69 


0.70 


Enriching Educational Experiences 


0.54 


0.64 


0.57 


0.53 


Supportive Campus Environment 


0.78 


0.78 


0.70 


0.70 



As can be seen from the table, measures of internal consistency in the benchmark scores are fair, 
but noticeably soft in two areas for NSSE and in three areas for [author’s institution]: Active and 
Collaborative Learning, Student-Faculty Interaction, and Enriching Educational Experiences. At 
[author’s institution], Cronbach’s alpha scores for the remaining benchmarks are minimally 
acceptable. 

A low alpha does not necessarily mean that the test or scale is bad per se, but rather that there is 
some concern about the cohesiveness of the scales. This may mean that the scale items 
insufficiently (or inefficiently) measure the target concept, or that they may be measuring more 
than one concept. While we will move forward with analysis on these scales, we expect to see 
that the additional error within the benchmarks may reduce the measurable association with the 
student outcomes. 

Explanatory Power of the NSSE Benchmarks 

To explore the association between engagement benchmarks and our student outcomes, an OLS 
Regression analysis was conducted using the benchmark scales. Using a benchmark-only model 
to predict GPA, a minor association was found between GPA and all five benchmarks, with an 
adjusted r-square of .023 for freshmen and .053 for seniors. Looking at individual benchmarks. 
Freshman GPA had three with significant, albeit minor contributions: Level of Academic 
Challenge, Active and Collaborative Learning, and Enriching Educational Experiences. 
Engagement showed a larger impact for seniors, with Student Faculty interaction and Enriching 
Educational Experiences having the strongest influence on GPA. 

Following the analysis run by Carini, Kuh, & Klein (2005), we tested a model controlling for 
several potentially confounding elements, generally reflecting pre-existing differences or 



While there are no absolute criteria in the literature, practitioners usually strive for .70 or above. The minimum level of internal 
consistency is to some extent dependent upon the goals and objectives of the scale. If a research wishes to be extremely precise in 
their measurements (such as IQ tests), much higher alpha levels are required. See Nunnally, J.C. (1978). Psychometric Theory (2nd 
ed.). New York: McGraw-Hill. 
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common predictors of success in college. 3 To avoid excessive R-square deflation and collinearity 
issues from multiple elements, these factors were combined into a single control variable. Even 
with this step, the condition index suggests a definite risk of collinearity. 4 With these factors 
controlled, the scales had a similar result pattern, with minimal impact from the engagement 
benchmarks. 



Table 2. Partial Correlations for Individual Benchmarks with GPA (Regression) 











Ft 2 for 


Adjusted 








Partial 


Partial 


R 2 for 


Cumulative GPA 


Benchmarks 


Std. Beta 


Corr. 


Corr. 


Model 


First- Year (Freshmen) 


Control Factor 


0.432 


0.437 


0.1910* 


0.208 




Level of Academic Challenge 


0.101* 


0.099 


0.0098 






Active and Collaborative Learning 


0.083* 


0.073 


0.0053 






Student-Faculty Interaction 


0.007 


0.006 


0.0000 






Enriching Educational Experiences 


-0.110* 


-0.109 


0.0119 






Supportive Campus Environment 


0.054 


0.057 


0.0032 




Seniors 


Control Factor 


0.454 


0.458 


0.2098* 


0.242 




Level of Academic Challenge 


-0.052 


-0.053 


0.0028 






Active and Collaborative Learning 


0.075 


0.070 


0.0049 






Student-Faculty Interaction 


0.080 


0.075 


0.0056 






Enriching Educational Experiences 


-0.005 


-0.005 


0.0000 






Supportive Campus Environment 


0.112* 


0.120 


0.0144 





* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 



Overall, the benchmarks provide minimal predictive value of academic performance as expressed 
through GPA. There is a stronger relationship for seniors than freshmen, suggesting that 
engagement has an influence over the career of the student — something not readily seen after a 
semester and a half of experience. It is also interesting to note that Enriching Educational 
Experiences have a significant and negative effect on freshman GPA (but no effect on senior 
GPA). A reasonable explanation would be that too positive an engaging experience may be 
detrimental early in a student’s career, distracting them from basic studies and adjusting to the 
[author’s institution] campus environment. This explanation is borne out by [author’s] (2000) 
findings that students in learning communities at [author’ s institution] were retained at a lower 
rate and exhibited lower performance (GPA) than control group students not involved in learning 
communities. 

Retention analysis focused on the freshmen respondents. Among freshmen, only 66 of the 1244 
respondents (5.3%) did not return for the sophomore year, thus (fortunately, from an institutional 
perspective) limiting the available sample size for this analysis at the outset. Our initial analysis 



Variables controlled included: Total SAT score, gender, ethnicity, NSSE 10-major group, parental education, commute distance 
and full-time/part-time status. 

The condition index is the square root of the ratio of the largest eigenvalue (the largest contributor) to the eigenvalue for each 
successive dimension (variable) added to the model. Levels over 15 suggest a risk of collinearity, and levels over 30 indicate a 
critical collinearity issue — variables effectively can collapse together. 
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showed that GPA has a significant association with retention. As a result, we added GPA to the 
model as an additional control valuable. 

Only one of the benchmarks exhibited significant beta coefficients in explaining whether or not a 
student was retained: Supportive Campus Environment. It should be noted that while statistically 
significant, the association was extremely weak, accounting for only 1 .0 percent of the variance. 



Table 3. Significant Contributors to Retention (Benchmarks) — All Students (N = 1083) 





Beta 


Partial Corr. 


Partial R 2 


Adjusted R' 
for Model 


Retained? (1 = No. 2 = Yes) 
Control Factor 


0.131* 


0.133 


0.0177* 


.080* 


GPA 


0.202* 


0.201 


0.0404* 




Level of Academic Challenge 


0.004 


0.003 


0.0000 




Active and Collaborative Learning 


0.059 


0.048 


0.0023 




Student-Faculty Interaction 


-0.067 


-0.056 


0.0031 




Enriching Educational Experiences 


-0.028 


-0.025 


0.0006 




Supportive Campus Environment 


0.104* 


0.100 


0.0100 





* Beta - significant individual contributor (p < .05) 

* R-square - small or greater effect size ( >.03) 



A total of 15 1 senior respondents to NSSE were matched to their responses to questions about 
their immediate employment and education plans on the [author’s institution] Commencement 
Survey. In this analysis, employment was operationalized as whether or not a student reported 
having accepted an offer of a full-time (35+ hours per week) employment at the time of 
commencement. Plans for further education consisted of a four-point set of choices, and was 
treated as a Likert-type scale of interest in this analysis 5 . Again, NSSE benchmarks made 
extremely modest contributions to predicting senior outcomes. Table 4 presents the results of the 
OLS regression models on the two senior outcomes. As can be seen from the table, none of the 
benchmarks are significantly related to whether or not a student reported having obtained a job 
upon commencement. In terms of further education, Student Faculty Interaction was significant in 
whether or not a senior planned to pursue further education. 



Future education plans is an ordinal scale of plans and behaviors regarding future education, and to an extent approximates a Likert- 
type interest scale. Given the poor predictive ability of our logistic models, and the more direct measures of effect size drawn from 
the OLS regressions, we felt that this would be a more fruitful mode for evaluating the measure-outcome links, at least in terms of 
the size and direction of the relationships. 
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Table 4. Significant Associations Between Benchmarks and Senior Outcomes 



Dependent Significant Contributors 

Variable (p < .05) 


Beta 


Partial 

Corr. 


R 2 for 
Partial 
Correlation 


Adjusted 
R 2 for 
Model 


Reported employment at time of graduation (0 = No, 1 = Yes) 








.247 


Control Factor 


0.496 


0.491 


0.2411* 




Level of Academic Challenge 


-0.151 


-0.158 


0.0250 




Active and Collaborative Learning 


0.088 


0.085 


0.0072 




Student-Faculty Interaction 


-0.015 


-0.014 


0.0002 




Enriching Educational Experiences 


0.122 


0.115 


0.0132 




Supportive Campus Environment 


0.023 


0.025 


0.0006 




Future Education Plans 

(1 = does not plan to, 2 = considering, 3 = applying, 4 = currently accepted) 






.313 


Control Factor 


0.498 


0.510 


0.2601* 




Level of Academic Challenge 


0.048 


0.052 


0.0027 




Active and Collaborative Learning 


0.049 


0.050 


0.0025 




Student-Faculty Interaction 


0.209* 


0.195 


0.0380* 




Enriching Educational Experiences 


-0.151 


-0.156 


0.0243 




Supportive Campus Environment 


0.063 


0.068 


0.0046 





* - Significant contributor to regression model ( p < .05) 



Taken overall, the NSSE benchmarks provide very little predictive power on the student 
outcomes of concern. However, as Carini, Kuh and Klein (2005) note, students of higher ability 
(specifically operationalized as a higher SAT score) benefit less from student engagement 
elements, compared to those of more median ability, [author’s institution] is a high-selectivity 
institution — the mean SAT total for incoming freshmen for Fall 2005 was 1340. Thus, it is 
possible that other institutions may find more significant relationships between the benchmarks 
and student outcomes. For [author’s institution], we believe further exploration of NSSE — 
beyond the benchmarks — is required. We now turn to an exploration of an alternative formulation 
of the benchmarks created by Pike (2004a), as well as a data-driven approach focusing on the 
specific NSSE items and their relationship to student outcomes at [author’s institution]. 



Pike’s Scalelets 

Pike (2004a) introduced an alternative grouping of NSSE items that he termed “scalelets.” The 
scalelets are a set of 12 derived scales — in many ways an extension and refinement of the NSSE 
benchmarks — designed primarily to provide more concise and actionable data for academic 
departments within institutions. It is possible that the tighter theoretical concepts as well as the 
larger number of items included in each scalelet might prove more effective at predicting student 
outcomes. 6 We recognize that the scalelets were developed as a means for describing results at 
the academic unit, not individual levels, and thus may not be optimal measures of individual 
student outcomes. 

As with the NSSE benchmarks, Cronbach’s alpha was used to measure the internal consistency of 
the scales. Compared to the NSSE benchmarks, the scalelets demonstrated less internal 
consistency in the analysis of [author’s institution] results. All of the scalelets have alpha 
coefficients below 0.70, many of them markedly so. The reliability of the scalelets for [author’s 
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The NSSE benchmarks (5 variables) contain 41 items, while Pike’s scalelets (12 variables) encompass 46 items. 




institution] students are reported in Table 5. The results for [author’s institution] are substantially 
similar to the alphas computed from Pike’s original data. 



Several explanations may play a role in these results. As Cronbach’s alpha can be directly 
influenced by the number of items involved in each scale, the relatively small number of items in 
each scalelet — eight of the twelve contain fewer than five items — may be a limiting factor. There 
may be problems with the formulation of specific benchmarks. Varied Experiences, for example, 
has the largest number of scale items of the scalelets, and as such might be expected to have a 
higher alpha, whereas in actuality it has one of the lowest alphas of the set. Almost by definition, 
the items that comprise this scalelet (such as community service, field experience/practicum, and 
study abroad), may contain too much variety to coalesce into a unified concept. 



Table 5. Scalelet Reliability ([author’s institution] 2005 Data) 









Freshmen 




Seniors 




Number of 




Cronbach's 




Cronbach's 


Scalelet 


Items 


N 


Alpha 


n 


Alpha 


Course challenge 


5 


mi 


0.527 


595 


0.526 


Writing 


5 


1151 


0.578 


602 


0.597 


Higher-Order Thinking Skills 


5 


1153 


0.626 


602 


0.642 


Active Learning 


3 


1179 


0.408 


606 


0.444 


Collaborative Learning 


4 


1160 


0.437 


605 


0.404 


Course Interaction 


3 


1159 


0.529 


605 


0.536 


Out of Class Interaction 


3 


1127 


0.413 


599 


0.554 


Varied Experiences 


7 


1116 


0.516 


595 


0.488 


Information Technology 


2 


1179 


0.578 


607 


0.527 


Diversity 


3 


1110 


0.633 


593 


0.646 


Support of Student Success 


3 


1110 


0.699 


595 


0.688 


Interpersonal Environment 


3 


1125 


0.614 


597 


0.645 



The alpha coefficients suggest that the utility of Scalelets for this sort of individual level 
evaluation may be somewhat limited. Nonetheless, we have chosen to pursue further analysis to 
determine whether the additional variables and the more granular typology of engagement 
encapsulated by Pike’s approach contribute to our understanding of engagement and student 
outcomes. 

As with the NSSE benchmarks, we ran an OLS regression using the same control variables, but 
substituted the Pike scalelets as independent variables. The full results of the regression models 
arc presented in the appendix. In the table below, we present only those variables with 
statistically significant beta coefficients, the direction of the relationship, and the partial r-square 
values for each significant variable. 
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Table 6. Significant Beta Coefficients for Pike Scalelets on Student Outcomes: Direction of 
Relationship and Partial R-Squared Values 7 





FYGPA 


FY Ret 


SRGPA 


SR job 


SR Edu 


Adjusted R-Square for model 


.221 


.082 


.274 


.272 


.308 


Control Variable 


+ * 


+ * 


+ * 


+ * 


+ * 


(.185) 


(.021) 


(.206) 


(.281) 


(.255) 


GPAf 




+ * 

(.037) 







Scalelets 



: Course challenge 


+ * 

(.016) 










Writing 












Higher-Order Thinking Skills 










+ * 

(.038) 


Active Learning 






+* 

(.009) 






j Collaborative Learning 












Course Interaction 












Out of Class Interaction 




_* 

(.005) 


+* 

(.019) 




+* 

(.044) 


Varied Experiences 




+ * 

(.005) 




+ 

(.035) 


1 


Information Technology 


_* 

(.006) 






+ 

(.033) 




Diversity 


_* 

(.011) 






(.036) 




Support of Student Success 












Interpersonal Environment 


+* 

(.004) 


+* 

(.004) 


+* 

(.014) 







+/- : direction of association (beta) 
t : Used in retention model only 
* : significant at p <. 05 



Given our available responses, the Pike scalelet models for the senior job and education outcomes are over-specified. Using 
Green’s (1991) rule of thumb, the 13-item scalelet model should have at least 154 respondents to test both the overall model and the 
individual scalelets within the model. In addition, the number of responses on the employment variable provided insufficient power 
to claim significance for the reported r-squares. An estimated 271 respondents would be needed to show significance on an r-square 
of .03. Although this limits the confidence in these specific measures, they are close enough to provide a general shape of the 
associations that might have been found in a larger sample. 
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Overall, the scalelets represent a very modest improvement over the NSSE benchmarks in 
accounting for academic performance. In predicting GPA, the scalelets provide a small, 
significant accounting of the variance, with an adjusted r-square of .043 for freshmen and .094 for 
seniors. While there are some scalelets that show significant individual contributions, none of 
them account for more than two percent of the variance in GPA. The scalelets prove to be 
insufficient for predicting freshman retention within the [author’ s institution] environment. While 
Out of Class Interaction, Varied Experiences and Interpersonal Environment were significant 
regressors, none of these factors accounted for any more than 0.5 percent of the variance. 8 

The scalelets performed better in predicting outcomes on employment and further education for 
graduating seniors. Three scalelets — Varied Experiences, Information Technology, and 
Diversity — were meaningful (but not statistically significant) contributors to whether or not the 
student reported having a job upon commencement. Two scalelets — Higher-Order Thinking 
Skills and Out of Class Interaction — were significantly related to the pursuit of a 
graduate/professional degree. Clearly, many of the academic and co-curricular activities related to 
interest in further education (such as working with faculty on research projects, discussing career 
plans with faculty) manifest in the decision to pursue post-baccalaureate education. 

While the scalelets are not as psychometrically reliable as the NSSE benchmarks, they do 
represent a modest improvement in predicting student outcomes at [author’s institution]. 
Unfortunately, a high risk of collinearity in a full scalelet model is nearly unavoidable, due to the 
fact that many of the scalelets are highly interrelated, particularly between those that arc derived 
from a similar NSSE benchmark. 9 



Back to the Basics: An Exploration of Individual NSSE Items 

Given the relatively modest contributions of both the NSSE benchmarks and the Pike scalelets to 
explaining student outcomes, as a next analytical step we chose to adopt a data-driven approach 
to explaining student outcomes at [author’s institution]. Our goal here was to determine exactly 
what individual NSSE items were related to student success as operationalized within this paper. 

We considered using exploratory factor analysis on the [author’s institution] data. Factor analysis 
would provide a new set of factors from the data, but the factors generated might not be 
predictive. As such, we felt it would be more sensible to begin by identifying predictors. Creating 
factors will by nature lose information. We required a predictive set of variables whose 
performance exceeds that of the benchmarks. Early tests of factor analysis models showed a 
drastic decline in explained variance from its constituent items, to the point of providing no better 
explanatory power than that produced by the benchmarks. We determined the models would 
perform better using the individual items rather than the factors. Therefore, our goal in this 
exercise was to use the data to create models that best explained our outcomes of interest, with 
the hope that common elements or patterns would be found among the NSSE variables. We then 
tested the model generated by the 2005 results on a previous cohort of NSSE respondents. 



It is interesting to note that the sign on Out of Class Interaction is negative — students who spend more time interacting with faculty 
outside of class were less likely to return. One possible explanation is that students who are considering leaving [author’s 
institution] may be engaging with faculty and advisors regarding alternative educational opportunities. We plan to investigate this 
relationship further in the future. 

While this risk of multicollinearity does suggest a weaker or unstable model, this is not immediately fatal. Others who wish to 
explore their data using the scalelets in this manner may try to include in the model only those scalelets that are of greatest 
importance or can be practically addressed by the institution. 
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We employed forward regression to maximize the explained variance by the models. This method 
starts with an empty model and adds variables, with the largest individual contributors added first. 
Additional items are added if they provide a significant increase in the predictive power (R- 
square) of the model. This approach differs from stepwise regression, in that stepwise regression 
will also remove items that fail to provide a significant contribution after additional variables are 
entered into the model (Tabachnick & Fiddel, 1996). The forward regression thus provides a 
more liberal pool of items by allowing individual contributors to remain, even if they become no 
longer significant due to intercorrelations with other items in the model. The main concern with 
this approach is the risk of over-specifying the model. 10 The results of the regression models arc 
presented in Table 7. 



Table 7. Significant Beta Coefficients for Individual NSSE Items on Student Outcomes: Direction of 
Relationship and Partial R-Squared Values 

Benchmark Key: 

LAC = Level of Academic Challenge 
ACL = Active and Collaborative Learning 
SFI = Student-Faculty Interaction 

EEE = Enriching Educational Experience FY FY SR SR SR 

SCE = Supportive Campus Environment Bench GPA Ret GPA Job Edu 



Control variable 




+ 

(.141) 


+ 

(.014) 


+ 

(.160) 


+ 

(.238) 


+ 

(.216) | 


GPA (For Freshman Retention only) 






+ 

(.028) 








Hours per 7-day week spent preparing for class (studying, 
reading, writing, doing homework or lab work, analyzing 
data, rehearsing, and other academic activities) 


LAC 


+ 

(.009) 










Coursework emphasized: MAKING JUDGMENTS about 
the value of information, arguments, or methods, such as 
examining how others gathered and interpreted data and 
assessing the soundness of their conclusions 


LAC 








(.184) 




Asked questions in class or contributed to class discussions 


ACL 






+ 

(.018) 






Tutored or taught other students (paid or voluntary ) 


ACL 


+ 

(.026) 




+ 

(.027) 






Discussed ideas from your readings or classes with faculty 
members outside of class 


SFI 


(.008) 











+/- : direction of association (sign of beta weight) 

items are statistically significant contributors to model (p < .05) 



In addition to being unwieldy, a model with a large number of predictors loses power for detecting significance among individual 
predictors, as well as for the model as a whole. This can be offset by having a sufficiently large sample of respondents. Over- 
specification can also contribute to problems with collinearity — individual predictor variables becoming highly overlapped, and 
fail to produce unique information — suggesting the removal of predictors may be needed. Initial trials using the GPA analysis 
indicated almost no difference between the stepwise and forward models for freshman GPA; the stepwise model dropped a single 
variable, but otherwise produced identical results. Also, for most of our exploratory analyses there was a sufficient subject pool to 
minimize the risks of power loss due to additional variables being considered in the models. 
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Table 7. Significant Beta Coefficients for Individual NSSE Items on Student Outcomes: Direction of 
Relationship and Partial R-Squared Values 

Benchmark Key: 

LAC = Level of Academic Challenge 
ACL = Active and Collaborative Learning 
SFI = Student-Faculty Interaction 

EEE = Enriching Educational Experience FY FY SR SR SR 

SCE = Supportive Campus Environment Bench GPA Ret GPA Job Edu 



Talked about career plans with a faculty member or 
advisor 


SFI 




(.006) 






+ 

(.120) | 


Work on a research project with a faculty member outside 
of course or program requirements 


SFI 






+ 

(.011) 






Used an electronic medium (listserv, chat group, Internet, 
instant messaging, etc.) to discuss or complete an 
assignment 


EEE 


(.010) 










Practicum, internship, field experience, co-op experience, 
or clinical assignment 


EEE 




+ 

(.005) 




+ 

(.067) 


(.086) 


Community service or volunteer work 


EEE 








+ 

(.103) 




Study abroad 


EEE 






+ 

(.013) 






Quality: Your relationships with other students 


SCE 




+ 

(.013) 








Quality: Your relationships with faculty members 


SCE 


(.006) 




(.056) 






Institutional emphasis: Providing the support you need to 
help you succeed academically 


SCE 


+ 

(.005) 










Institutional emphasis: Helping you cope with your non- 
academic responsibilities (work, family, etc.) 


SCE 


(.014) 










Come to class without completing readings or assignments 




(.020) 










Mark the box that best represents the extent to which your 
examinations during the current school year challenged 
you to do your best work. 




(.009) 











+/- : direction of association (sign of beta weight) 

items are statistically significant contributors to model (p < .05) 
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Table 7. Significant Beta Coefficients for Individual NSSE Items on Student Outcomes: Direction of 
Relationship and Partial R-Squared Values 

Benchmark Key: 

LAC = Level of Academic Challenge 
ACL = Active and Collaborative Learning 
SFI = Student-Faculty Interaction 

EEE = Enriching Educational Experience FY FY SR SR SR 

SCE = Supportive Campus Environment Bench GPA Ret GPA Job Edu 



Put together ideas or concepts from different courses when 
completing assignments or during class discussions 












+ 

(.070) I 


Number of problem sets (problem-based homework 
assignments) that take you MORE than an hour to 
complete 




(.005) 




(.005) 






Exercised or participated in physical fitness activities 




(.004) 










Tried to better understand someone else's views by 
imagining how an issue looks from his or her perspective 








(.021) 






Hours per 7-day week spent relaxing and socializing 
(watching TV, partying, etc.) 




(.006) 










Institutional emphasis: Attending campus events and 
activities (special speakers, cultural performances, athletic 
events, etc.) 






+ 

(.007) 








Institutional contribution: Acquiring job or work-related 
knowledge and skills 






8 + 






1 


Institutional contribution: Working effectively with others 










+ 

(.092) 




Institutional contribution: Learning effectively on your 
own 




+ 

(.006) 











+/- : direction of association (sign of beta weight) 

items are statistically significant contributors to model (p < .05) 



There are several findings worthy of note in these results. First, while several items that are found 
in the NSSE benchmarks are also significant variables in these models, there are quite a few that 
are not. The benchmarks encompass 41 items. Of these items, only 15 were found to contribute to 
the student outcomes in one way or another. The remaining 26 benchmark items made no 
significant explanatory contribution. Secondly, there are an additional 1 1 items that made a 
significant contribution to the models’ explanatory power that are not found in the benchmarks. 
Freshman GPA is the outcome best served by the additional items — six of the additional 1 1 items 
were found to contribute an additional five percent of the explained variance in this outcome. 
Third, there are few individual items that provide explanatory power across the several outcomes 
modeled. For example, only one item (participation in a practicum, internship, field experience, 
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co-op experience, or clinical assignment) provided explanatory power in freshman retention, 
senior job attainment and senior further education. No other single item was a significant 
explanatory factor in more than two of the measured outcomes. 

The results of the “best fit” models have generated some interesting and surprising findings for us 
as well. For example, student-faculty interaction in terms of discussion of readings/ideas with 
faculty outside of class is negatively related to freshman GPA, as is the student-reported 
institutional emphasis on helping students cope with non-academic responsibilities. It is possible 
that students who find themselves in academic or personal difficulties during their first year are 
more likely to engage in discussions with Institute support personnel like advisors or student 
affairs counselors. This might explain the negative sign of the beta coefficient in this case. While 
we were pleased to note that study abroad experiences were positively related to senior GPA, we 
were peiplexed at the negative correlation between senior GPA and ratings of faculty-student 
relationships. We also note the large and negative relationship between seniors reporting 
coursework emphasizing making judgments about the value of information and data 
interpretation and obtaining a job upon commencement. Given that student major is being 
controlled in this model, the fact that this single item explains over 1 8 percent of the observed 
variance definitely merits further investigation. 



Table 8. Marginal Improvements in Explanatory Power: Adjusted partial R-squared Contributions to 
Student Outcomes 





Cumulative GPA 
(Freshman) 


Cumulative GPA 
(Seniors) 


Freshman 

Retention 


Senior Job 
Attainment 


Senior Further 
Education 


Control Variable 


.184 


.216 


.067 


.256 


.280 


NSSE 

Benchmarks 


.024 


.026 


.009 


-.009 


.033 


Pike Scalelets 


.037 


.058 


.010 


.016 


.028 


Stepwise 
Regression of 
individual NSSE 
items 


.101 


.114 


.035 


.228 


.138 



Table 8 shows the relative contributions to explained variance of the student outcomes of the 
NSSE benchmarks, Pike’s scalelets, and the stepwise regression models on individual NSSE 
items. As can be seen, the control variables provide a largest amount of explanatory power. 
Flowever, it also notable the individual NSSE items do a much better job of explaining the 
student outcomes than do the benchmarks or scalelets. For example, for freshman GPA, the 
benchmarks explain an additional 2.4 percent of variance over the control variable. The stepwise 
regression model on individual NSSE items explains an additional 10.1 percent of variance. The 
regression model does particularly well explaining job attainment upon commencement with the 
selected NSSE items accounting for 22.8 percent of the variance over and above the controls. 
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Testing the Models on NSSE 2003 

In order to determine whether the results are stable within our institution, we decided to test the 
models against another student cohort — respondents to the 2003 NSSE. From the 2005 regression 
equations, we calculated predicted outcome variables based on 2003 responses. The 2003 NSSE 
was administered to only a sample of enrolled students, resulting in a smaller pool of respondents 
(N=641 versus N=1873 in 2005). Because of the interaction of sample size and item count, we 
realized from the beginning that the adjusted R-squares would be markedly lower for 2003. Our 
decision to proceed was prompted by the notion that this potential reduction in predictive power 
would function as a stringent test of the models. 

Some adjustments in the coding of the items were made due to differences in the wording of 
several NSSE items during this period. 11 Due to the smaller sample size, we were not able to test 
the model on senior job attainment nor senior further education plans. Table 9 compares the 
performance of the regression models for the 2003 and 2005 cohorts. For both years, the overall 
R-square is presented, along with the marginal contributions in explained variance for the control 
variables and the NSSE items generated from the 2005 data. As can be seen, the model seems to 
hold up fairly well when used to predict 2003 freshman GPA. The model, which explained 10.5 
percent of the variance in the 2005 GPA, explained 8.4 percent of the variance for the 2003 
freshman cohort. However, the model fared substantially worse in explaining senior GPA. The 
2005 model explained 1 1.4 percent of senior GPA variance, but in 2003, the adjusted R-square 
was negative (-.015) — essentially, the NSSE valuables simply introduced more noise into the 
equation. Freshman retention, which was not very well modeled in 2005, also had a negative 
adjusted R-square value in the 2003 model. 

The failure of two of the models generated in 2005 to work with 2003 (senior GPA and freshman 
retention) data is troubling. Of particular concern is the failure of the model to explain senior 
GPA. The control variables did a better job explaining variance in 2003 than did the NSSE items. 
While it is possible that some there is some environmental difference between the two samples, 
we know of no major institutional changes that might account for the differential performance of 
the models. We plan to attempt a similar analysis on data collected in our next NSSE 
administration (tentatively set for Spring 2007). However, our tentative conclusion is that the 
NSSE variables identified in the models have a decidedly mixed level of validity within our 
institution. 



In 2005, NSSE added some additional affirmative response choices to the variety of co-curricular and extra-curricular experiences 
items. We chose to modify the 2003 responses by re- weighting the affirmative responses based on the ratio of affirmative 
responses in 2005. Items that appeared in 2005 but not in 2003 were eliminated from the 2005 model for the purposes of 
generating the appropriate beta coefficients for the 2003 model. These items included how often a student: “tried to better 
understand someone else's views by imagining how an issue looks from his or her perspective” and how frequently a student 
“learned something that changed the way you understand an issue or concept.” 
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Table 9. Testing the 2005 Models: Adjusted Partial R-squared Contributions to Student Outcomes for 
2003 and 2005 [author’s institution] NSSE Data 





2003 


2005 


Freshman GPA 


N=350 


N=1074 


Model R-Square 


.211 


.284 


Control Adjusted R-Square 


.127 


.179 


Regression Variable contribution (18 items) 


.084 


.105 


Senior GPA 


N=214 


N=461 


Model R-Square 


.242 


.309 


Control Adjusted R-Square 


.257 


.195 


Regression Variable contribution (11 items) 


-.015 


.114 


Freshmen Retention 


N=350 


N=1074 


Model R-Square 


.016 


.102 


Control + GPA Adjusted R-Square 


.080 


.067 


Regression Variable contribution (10 items ) 


-.064 


.035 



Discussion 

Our goals in this project were to investigate the relationship between NSSE benchmark scores 
and student outcomes at [author’s institution], as well as further illuminate the ways in which 
NSSE can be used to describe and explain student success. We continue to believe that the 
National Survey of Student Engagement is a valuable tool for institutional self-reflection. 
However, we also recognize the limitations of the NSSE benchmarks as indicators of 
effectiveness. While there is some linkage between items that comprise the benchmarks and our 
operationalized outcomes, there exists notable room for improvement. We believe that an 
approach that eschews domain scales in favor of targeting the specific elements that are most 
directly related to outcomes offers superior explanatory power and predictive validity. However, 
even this approach has mixed results. While the model generated fairly stable results in 
explaining freshman GPA, it did not provide such stability for freshman retention or senior GPA. 

Another potential explanation for our findings may lie in the nature of the comparisons being 
pursued. As Pike (1996) demonstrated, there exists an ambiguous relationship between self- 
reported learning outcomes and objective test results. By extrapolation, the use of NSSE items to 
predict objective student outcomes may be a similar situation, and it may be that the low 
predictive power of self-reports to gauge student outcomes all but precludes their serious usage in 
this vein. 

We recognize that the results obtained here are not generalizable, and we encourage other 
institutions to conduct similar analyses. Through such investigations, institutions can get beyond 
the basic numbers to more thoroughly specify the nature of engagement within the college 
environment. Institutions must determine for themselves what policy changes are warranted from 
the data. Revealing the relationship between what students tell us through NSSE and how this 
relates (or does not relate) to issues such as academic performance, retention, persistence and 
future career success is a vital next step in the scholarship of student engagement. 

Putting the NSSE benchmarks to the empirical test is particularly important given the political 
and policy pressures that exist on disclosure of institutional performance on student surveys such 
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as the NSSE. It is quite understandable that the public we serve desires accurate information 
about the nature of the educational experience provided by institutions as well as measurable 
indicators of student success. However, we are skeptical that the NSSE benchmarks provide this 
service. In his foreword to the 2005 NSSE Annual Survey Results, Russ Edgerton argues that the 
NSSE benchmarks could be viewed as measures of institutional traits — similar perhaps to the 
Myers-Briggs personality inventory. However, unlike the popular personality profile, the 
benchmarks provide a quantitative measure of these traits. Edgerton believes that these 
benchmark values should be publicized as an institutional scorecard to bolster their reputations 
for “effectiveness.” Yet, we caution that one should beware of “benchmark blinders” — the 
assumption that the benchmarks provide a reliable and scalable index of comparison among 
institutions. One must consider the issue of meaningful scales. How much “better” or “more 
effective” is an institution that scores 65 on the Level of Academic Challenge Benchmark than 
one that scores a 60? 

Furthermore and perhaps more important, the analyses conducted here and elsewhere (Zhao and 
Kuh, 2004; Carini, Kuh and Klein, 2006) call into question the conclusion that the benchmarks 
are a valid and reliable indicator of institutional effectiveness. As Pike (1996: 111) aptly cautions, 
“A very real danger is that a public hungry for simple answers to complex questions will forget 
that self-reports of learning and academic development arc not precisely the same as more 
traditional measures of the same outcomes, and draw erroneous conclusions about the quality and 
effectiveness of postsecondary education.” If nothing else, the analysis suggests that using the 
benchmark scales would produce less informative results than the items selected from them 
through regression analysis. These types of measures may be of greater importance and utility 
than the benchmarks, and should be addressed by institutions, at least for internal evaluation. In 
the end, we believe that the benchmarks are best utilized as descriptions of character, not ability. 
If institutions are to use NSSE benchmarks in a public way to distinguish themselves from others, 
it would be far better to remove the quantitative aspects of the scales from the discourse and 
instead focus on how the benchmarks describe the qualitative characteristics of the educational 
experience that is a resultant of the interaction between the personality of the student and the 
personality of the institution. 
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Appendix 



NSSE Benchmark Models 

Freshman GPA (Cumulative GPA, Spring 2005) 

Model Summary 

Change Sta tistics 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.430 


0.185 


0.184 


0.619 


0.185 


245.482 


1 


1080 


< .001 


2 


.461 


0.212 


0.208 


0.6101 


0.027 


7.444 


5 


1075 


< .001 



1: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable), Enriching Educational Experiences, Supportive Campus 
Environment , Level of Academic Challenge (adjusted), Student-Faculty Interaction, Active & Collaborative Learning 



Partial correlations for individual Benchmarks with GPA (regression) 



Benchmarks 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


First- Year (Freshmen) 

Control Factor 


0.432* 


0.437 


0.1910* 


0.208* 


Level of Academic Challenge 


0.101* 


0.099 


0.0098 




Active and Collaborative Learning 


0.083* 


0.073 


0.0053 




Student-Faculty Interaction 


0.007 


0.006 


0.0000 




Enriching Educational Experiences 


-0.110* 


-0.109 


0.0119 




Supportive Campus Environment 


0.054 


0.057 


0.0032 




* for Beta: Significant contributor to regression model (p < .05) 

* for R- square: small effect size or greater (>.03) 









Senior GPA (Cumulative GPA, Spring 2005) 
Model Summary 



Change Statis tics 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.467 


0.218 


0.216 


0.46378 


0.218 


128.574 


1 


462 


< 0.001 


2 


.502 


0.252 


0.242 


0.45595 


0.034 


4.203 


5 


457 


0.001 



1: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable), Enriching Educational Experiences, Supportive Campus 
Environment , Level of Academic Challenge (adjusted), Student-Faculty Interaction, Active & Collaborative Learning 
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Appendix 



NSSE Benchmark Models (cont’d) 

Senior GPA (Cumulative GPA, Spring 2005) (cont’d) 



Benchmarks 


Std. Beta 


Partial 

Correlation 


Ft 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Seniors 








0 . 242 * 


Control Factor 


0.454* 


0.458 


0.2098* 




Level of Academic Challenge 


-0.052 


-0.053 


0.0028 




Active and Collaborative Learning 


0.075 


0.070 


0.0049 




Student-Faculty Interaction 


0.080 


0.075 


0.0056 




Enriching Educational Experiences 


-0.005 


-0.005 


0.0000 




Supportive Campus Environment 


0.112* 


0.120 


0.0144 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 









Freshmen Retention 



Model Summary 



Change Sta tistic s 



Model 


R 

R Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.169 


.028 


.028 


.210 


.028 


31.46 


1 


1073 


< 0.001 


2 


.262 


.069 


.067 


.205 


.04 


46.239 


1 


1072 


< 0.001 


3 


.286 


.082 


.076 


.204 


.013 


3.033 


5 


1067 


0.01 



1: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value) 

2: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value), Spring GPA (cumulative) 

3: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value), Spring GPA (cumulative). Enriching Educational 
Experiences, Supportive Campus Environment , Level of Academic Challenge (adjusted), Student-Faculty Interaction, Active and 
Collaborative Learning 





Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Retained? (1= No, 2 = Yes) 








.076* 


Control Factor 


0.128* 


0.130 


0.0168 




GPA 


0.192* 


0.191 


0.0365* 




Benchmarks 










Level of Academic Challenge 


0.006 


0.005 


0.0000 




Active and Collaborative Learning 


0.062 


0.050 


0.0025 




Student-Faculty Interaction 


-0.071 


-0.059 


0.0034 




Enriching Educational Experiences 


-0.029 


-0.026 


0.0007 




Supportive Campus Environment 


0.106* 


0.102 


0.0103 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 
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NSSE Benchmark Models (cont’d) 

Senior: Job Placement at Graduation 



Model Summary 



Change Statistic s 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.514 


.264 


.256 


.394 


.264 


33.705 


1 


94 


< 0.001 


2 


.542 


.294 


.247 


.397 


.03 


0.764 


5 


89 


0.578 



1: Predictors: (Constant), Employment Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), Employment Control (Unstandardized Predicted Variable), Enriching Educational Experiences, Supportive Campus 
Environment , Level of Academic Challenge (adjusted), Student-Faculty Interaction, Active & Collaborative Learning 







Partial 


R 2 for Partial 


Adjusted R 2 for 


Benchmarks 


Std. Beta 


Correlation 


Correlation 


Model 


Reported employment at time of graduation (0 


= No, 1 = Yes) 






.247* 


Control Factor 


0.496* 


0.491 


0.2411* 




Level of Academic Challenge 


-0.151 


-0.158 


0.0250 




Active and Collaborative Learning 


0.088 


0.085 


0.0072 




Student-Faculty Interaction 


-0.015 


-0.014 


0.0002 




Enriching Educational Experiences 


0.122 


0.115 


0.0132 




Supportive Campus Environment 


0.023 


0.025 


0.0006 





Senior: Plans For Further Education 
Model Summary 



Change Statistics 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.535 


.287 


.280 


.801 


.287 


47.389 


l 


118 


< 0.001 


2 


.590 


.348 


.313 


.782 


.061 


2.124 


5 


113 


0.068 



1: Predictors: (Constant), Continuing Ed Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), Continuing Ed Control (Unstandardized Predicted Variable), Enriching Educational Experiences, Supportive 
Campus Environment , Level of Academic Challenge (adjusted), Student-Faculty Interaction, Active & Collaborative Learning 







Partial 


R 2 for Partial 


Adjusted R 2 for 


Benchmarks 


Std. Beta 


Correlation 


Correlation 


Model 


Future Education Plans 

( 1 = does not plan to, 2 = considering, 3 = applying, 4 


= currently accepted) 






.313* 


Control Factor 


0.498* 


0.51 


0.2601* 




Level of Academic Challenge 


0.048 


0.052 


0.0027 




Active and Collaborative Learning 


0.049 


0.05 


0.0025 




Student-Faculty Interaction 


0.209* 


0.195 


0.0380* 




Enriching Educational Experiences 


-0.151 


-0.156 


0.0243 




Supportive Campus Environment 


0.063 


0.068 


0.0046 
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Appendix 



NSSE — Scalelets 



Freshman GPA (Cumulative GPA, Spring 2005) 
Model Summary 



Change Statistics 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.430 


.185 


.184 


.61914 


.185 


245.482 


1 


1080 


< 0.001 


2 


.480 


.231 


.221 


.60495 


.046 


5.271 


12 


1068 


< 0.001 



1: Predictors: (constant) GPA control (Unstandardized Predicted Value)... 

2: Predictors: (Constant), GPA control (Unstandardized Predicted Value), Support of Student Success, Writing, Varied Experiences, Higher- 
Order Thinking Skills, Information Technology, Course challenge, Interpersonal Environment, Diversity, Out of Class Interaction, Active 
Learning, Collaborative Learning, Course Interaction 



Scalelet 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


First- Year (Freshmen) 








.221* 


Control Factor 


0.427* 


0.430 


0.1849* 




Course challenge 


0.125* 


0.128 


0.0164 




Writing 


-0.014 


-0.013 


0.0002 




Higher-Order Thinking Skills 


0.053 


0.052 


0.0027 




Active Learning 


0.060 


0.056 


0.0031 




Collaborative Learning 


0.062 


0.056 


0.0031 




Course Interaction 


-0.010 


-0.008 


0.0001 




Out of Class Interaction 


0.023 


0.021 


0.0004 




Varied Experiences 


0.014 


0.014 


0.0002 




Information Technology 


-0.078* 


-0.074 


0.0055 




Diversity 


-0.109* 


-0.106 


0.0112 




Support of Student Success 


0.005 


0.005 


0.0000 




Interpersonal Environment 


0.061* 


0.061 


0.0037 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 
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NSSE — Scalelets (cont’d) 



Senior GPA (Cumulative GPA, Spring 2005) 

Model Summary 

Change S tatistics 



Adjusted R Std. Error of the r Square Sig. F 



Model 


R 


R Square 


Square 


Estimate 


Change 


F Change 


dfl 


df2 


Change 


1 


.467 


.218 


.216 


.46378 


.218 


128.574 


1 


462 


< 0.001 


2 


.543 


.295 


.274 


.44626 


.077 


4.082 


12 


450 


< 0.001 



1: Predictors: (constant) GPA control (Unstandardized Predicted Value)... 

2: Predictors: (Constant), GPA control (Unstandardized Predicted Value), Support of Student Success, Writing, Varied Experiences, Higher- 
Order Thinking Skills, Information Technology, Course challenge, Interpersonal Environment, Diversity, Out of Class Interaction, Active 
Learning, Collaborative Learning, Course Interaction. 



Scalelet 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Seniors 








.274* 


Control Factor 


0.447* 


0.454 


0.2061* 




Course challenge 


0.067 


0.072 


0.0052 




Writing 


-0.026 


-0.026 


0.0007 




Higher-Order Thinking Skills 


-0.066 


-0.068 


0.0046 




Active Learning 


0.098* 


0.096 


0.0092 




Collaborative Learning 


0.033 


0.032 


0.0010 




Course Interaction 


-0.046 


-0.042 


0.0018 




Out of Class Interaction 


0.144* 


0.139 


0.0193 




Varied Experiences 


0.037 


0.041 


0.0017 




Information Technology 


-0.079 


-0.081 


0.0066 




Diversity 


-0.085 


-0.091 


0.0083 




Support of Student Success 


0.02 


0.021 


0.0004 




Interpersonal Environment 


0.118* 


0.12 


0.0144 




* for Beta: Significant contributor to regression model (p < .05) 

* for R- square: small effect size or greater (>.03) 
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NSSE — Scalelets (cont’d) 



Freshmen Retention 
Model Summary 



Change Statistics 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.169 


.028 


.028 


.21 


.028 


31.46 


1 


1073 


<0.001 


2 


.262 


.069 


.067 


.205 


.04 


46.239 


1 


1072 


< 0.001 


3 


.299 


.089 


.077 


.204 


.021 


2.014 


12 


1060 


0.02 



1: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value) 

2: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value), Spring GPA (cumulative) 

3: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value), Spring GPA (cumulative), Support of Student Success, 
Writing, Varied Experiences, Higher-Order Thinking Skills, Information Technology, Course challenge, Interpersonal Environment, 
Diversity, Out of Class Interaction, Active Learning, Collaborative Learning, Course Interaction 





Std. Beta 


Partial Corr. 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Retained? (1 = No, 2 = Yes) 
Control Factor 


0.143* 


0.144 


0.0207 


.082* 


GPA 


0.196* 


0.193 


0.0372* 




Scalelets 


Course challenge 


-0.01 


-0.009 


0.0001 




Writing 


0.001 


0.001 


0.0000 




Higher-Order Thinking Skills 


0.034 


0.031 


0.0010 




Active Learning 


0.048 


0.041 


0.0017 




Collaborative Learning 


0.02 


0.016 


0.0003 




Course Interaction 


-0.007 


-0.005 


0.0000 




Out of Class Interaction 


-0.085* 


-0.073 


0.0053 




Varied Experiences 


0.072* 


0.068 


0.0046 




Information Technology 


-0.035 


-0.031 


0.0010 




Diversity 


-0.046 


-0.042 


0.0018 




Support of Student Success 


0.054 


0.049 


0.0024 




Interpersonal Environment 


0.066* 


0.061 


0.0037 
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NSSE — Scalelets (cont’d) 



Senior: Job Placement at Graduation 



Change Sta tistic s 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.514 


.264 


.256 


.394 


.264 


33.705 


l 


94 


< 0.001 


2 


.610 


.372 


.272 


.39 


.108 


1.176 


12 


82 


0.314 



1: Predictors: (constant) Employment control(Unstandardized Predicted Value) 

2: Predictors: (Constant), Employment control (Unstandardized Predicted Value), Support of Student Success, Writing, Varied Experiences, 
Higher-Order Thinking Skills, Information Technology, Course challenge, Interpersonal Environment, Diversity, Out of Class Interaction, 
Active Learning, Collaborative Learning, Course Interaction 



Scalelet 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Reported employment at time of graduation (0 


= No, 1 = Yes) 






.272* 


Control Factor 


0.521 


0.53 


0.2809* 




Course challenge 


0.039 


0.039 


0.0015 




Writing 


0.007 


0.008 


0.0001 




Higher-Order Thinking Skills 


-0.159 


-0.157 


0.0246 




Active Learning 


0.086 


0.084 


0.0071 




Collaborative Learning 


0.083 


0.079 


0.0062 




Course Interaction 


-0.116 


-0.103 


0.0106 




Out of Class Interaction 


0.008 


0.008 


0.0001 




Varied Experiences 


0.179 


0.188 


0.0353* 




Information Technology 


0.168 


0.182 


0.0331* 




Diversity 


-0.169 


-0.19 


0.0361* 




Support of Student Success 


0.134 


0.147 


0.0216 




Interpersonal Environment 


-0.079 


-0.084 


0.0071 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 
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NSSE — Scalelets (cont’d) 



Senior: Plans For Further Education 



Model Summary 



Change Statistics 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of the 
Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.535 


.287 


.280 


.801 


.287 


47.389 


1 


118 


< 0.001 


2 


.619 


.383 


.308 


.785 


.097 


1.386 


12 


106 


0.184 



1: Predictors: (constant) Continuing Ed control(Unstandardized Predicted Value) 

2: Predictors: (Constant), Continuing Ed control (Unstandardized Predicted Value), Support of Student Success, Writing, Varied Experiences, 
Higher-Order Thinking Skills, Information Technology, Course challenge, Interpersonal Environment, Diversity, Out of Class Interaction, 
Active Learning, Collaborative Learning, Course Interaction 



Scalelet 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Future Education Plans 

( 1 = does not plan to, 2 = considering, 3 = applying, 4 


= currently accepted) 






.308* 


Control Factor 


0.505* 


0.505 


0.2550* 




Course challenge 


-0.105 


-0.110 


0.0121 




Writing 


-0.024 


-0.025 


0.0006 




Higher-Order Thinking Skills 


0.193* 


0.195 


0.0380* 




Active Learning 


-0.049 


-0.047 


0.0022 




Collaborative Learning 


0.069 


0.067 


0.0045 




Course Interaction 


0.059 


0.053 


0.0028 




Out of Class Interaction 


0.217* 


0.210 


0.0441* 




Varied Experiences 


-0.079 


-0.087 


0.0076 




Information Technology 


-0.045 


-0.050 


0.0025 




Diversity 


-0.081 


-0.092 


0.0085 




Support of Student Success 


0.001 


0.001 


0.0000 




Interpersonal Environment 


0.07 


0.075 


0.0056 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 
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Regression Models 



Freshman GPA (Cumulative GPA, Spring 2005) 

Model Summary 

Change S tatistics 



Adjusted R Std. Error of the r Square Sig. F 



Model 


R 


R Square 


Square 


Estimate 


Change 


F Change 


dfl 


df2 


Change 


1 


.433 


.188 


.187 


.616 


.188 


248.291 


1 


1074 


<0.001 


2 


.549 


.301 


.288 


.577 


.114 


9.039 


19 


1055 


<0.001 



1: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

Tutored or taught other students 

Come to class without completing readings or assignments 

Institutional emphasis: Helping you cope with your non-academic responsibilities 

Quality: Your relationships with faculty members 

Discussed ideas from your readings or classes with faculty members outside of class 
Institutional contribution: Learning effectively on your own 
Used an electronic medium to discuss or complete an assignment 

The extent to which your examinations during the current school year challenged you to do your best work 
Hours per 7-day week spent preparing for class 

Number of problem sets (problem-based homework assignments) that take you MORE than an hour to complete 

Worked with faculty members on activities other than course work 

Institutional contribution: Acquiring job or work-related knowledge and skills 

Hours per 7-day week spent participating in co-curricular activities 

Institutional emphasis: Providing the support you need to help you succeed academically 

Exercised or participated in physical fitness activities 

Hours per 7-day week spent relaxing and socializing 

Asked questions in class or contributed to class discussions 

Number of written papers or reports of 20 PAGES OR MORE 

Work on a research project with a faculty member outside of course or program requirements 



Variables 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Freshmen 








. 288 * 


Control Factor 


0.359* 


0.376 


0.1412* 




Tutored or taught other students 


0.147* 


0.161 


0.0260 




Come to class without completing readings 
or assignments 


-0.127* 


-0.142 


0.0200 




Institutional emphasis: Helping you cope 
with your non-academic responsibilities 


-0.114* 


-0.119 


0.0141 




Quality: Your relationships with faculty 
members 


0.076* 


0.079 


0.0063 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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Regression Models (cont’d) 

Freshman GPA (Cumulative GPA, Spring 2005) (cont’d) 



Variables 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Discussed ideas from your readings or 
classes with faculty members outside of 
class 


-0.086* 


-0.088 


0.0077 




Institutional contribution: Learning 
effectively on your own 


0.074* 


0.080 


0.0064 




Used an electronic medium to discuss or 
complete an assignment 


-0.086* 


-0.098 


0.0096 




The extent to which your examinations 
during the current school year challenged 
you to do your best work 


-0.081* 


-0.090 


0.0080 




Hours per 7-day week spent preparing for 
class 


0.085 


0.093 


0.0086 




Number of problem sets (problem-based 
homework assignments) that take you 
MORE than an hour to complete 


-0.062* 


-0.069 


0.0048 




Worked with faculty members on activities 
other than coursework 


0.067* 


0.072 


0.0052 




Institutional contribution: Acquiring job 
or work-related knowledge and skills 


0.060* 


0.065 


0.0042 




Hours per 7-day week spent participating in 
co-curricular activities 


-0.013 


-0.015 


0.0002 




Institutional emphasis: Providing the 
support you need to help you succeed 
academically 


0.068* 


0.069 


0.0048 




Exercised or participated in physical 
fitness activities 


-0.058* 


-0.065 


0.0042 




Hours per 7-day week spent relaxing and 
socializing 


-0.067* 


-0.075 


0.0057 




Asked questions in class or contributed to 
class discussions 


0.056 


0.060 


0.0036 




Number of written papers or reports of 20 
PAGES OR MORE 


-0.031 


-0.035 


0.0012 




Work on a research project with a faculty 
member outside of course or program 
requirements 


0.028 


0.032 


0.0010 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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Regression Models (cont’d) 



Senior GPA (Cumulative GPA, Spring 2005) 
Model Summary 



Chan ge Statistics 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of the 
Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.467 


.218 


.217 


.463 


.218 


128.744 


1 


461 


<0.001 


2 


.593 


.351 


.331 


.428 


.133 


7.073 


13 


448 


<0.001 



1: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

2: Predictors: (Constant), GPA Control (Unstandardized Predicted Variable) 

Work on a research project with a faculty member outside of course or program requirements 

Participated in Study abroad 

Quality: Your relationships with faculty members 

Hours per 7 -day week spent working for pay OFF CAMPUS 

Tutored or taught other students 

Practicum, internship, field experience, co-op experience, or clinical assignment 

Tried to better understand someone else's views by imagining how an issue looks from his or her perspective 
Number of problem sets (problem-based homework assignments) that take you MORE than an hour to complete 
Asked questions in class or contributed to class discussions 
Discussed grades or assignments with an instructor 

Learned something that changed the way you understand an issue or concept 

Hours per 7-day week spent working for pay ON CAMPUS 

Discussed ideas from your readings or classes with others outside of class 
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Regression Models (cont’d) 



Senior GPA (Cumulative GPA, Spring 2005) (cont’d) 



Variables 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Seniors 








.331* 


Control Factor 


0.402* 


0.400 


0.1603* 




Work on a research project with a faculty 
member outside of course or program 
requirements 


0.090* 


0.105 


0.0110 




Participated in Study abroad 


0.097* 


0.115 


0.0132 




Quality: Your relationships with faculty 
members 


0.207* 


0.237 


0.0563* 




Hours per 7-day week spent working for 
pay OFF CAMPUS 


-0.003 


-0.003 


0.0000 




Tutored or taught other students 


0.142* 


0.165 


0.0271 




Practicum, internship, field experience, co- 
op experience, or clinical assignment 


0.070 


0.083 


0.0069 




Tried to better understand someone else's 
views by imagining how an issue looks 
from his or her perspective 


-0.141* 


-0.143 


0.0205 




Number of problem sets (problem-based 
homework assignments) that take you 
MORE than an hour to complete 


-0.057 


-0.068 


0.0046 




Asked questions in class or contributed to 
class discussions 


0.118* 


0.134 


0.0178 




Discussed grades or assignments with an 
instructor 


-0.049 


-0.056 


0.0032 




Learned something that changed the way 
you understand an issue or concept 


0.049 


0.048 


0.0023 




Hours per 7-day week spent working for 
pay ON CAMPUS 


-0.070 


-0.082 


0.0068 




Discussed ideas from your readings or 
classes with others outside of class 


-0.073 


-0.079 


0.0063 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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Regression Models (cont’d) 

Freshmen Retention 

Model Summary 



Change Sta tistic s 



Model 


R 


R 

Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.169 


.028 


.028 


.21 


.028 


31.46 


1 


1073 


< 0.001 


2 


.262 


.069 


.067 


.205 


.04 


46.239 


1 


1072 


<0.001 


3 


.334 


.112 


.102 


.202 


.043 


5.152 


10 


1062 


<0.001 



1: Predictors: (Constant), Control - Fr. Retained Unstandardized Predicted Value 

2: Predictors: (Constant), Control - Fr. Retained Unstandardized Predicted Value, Spring GPA (cumulative) 

3: Predictors: (Constant), Control - Retained (Unstandardized Predicted Value), Spring GPA (cumulative) 

Talked about career plans with a faculty member or advisor 

Institutional emphasis: Attending campus events and activities (special speakers, cultural performances, athletic events, etc.) 
Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or 
personal values 

Practicum, internship, field experience, co-op experience, or clinical assignment 
Institutional contribution: Acquiring job or work-related knowledge and skills 
Hours per 7 -day week spent working for pay OFF CAMPUS 
Quality: Your relationships with other students 

Received prompt feedback from faculty on your academic performance (written or oral) 

Tutored or taught other students (paid or voluntary) 

Institutional contribution: Understanding people of other racial and ethnic backgrounds 



Variables 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Reported employment at time of graduation (0 = No, 1 = Yes) 




.102* 


Control - Fr. Retained Unstandardized 
Predicted Value 


! 

0.120* i 0.119 


0.0142 




Spring GPA (cumulative) 


0.167* 


0.166 


0.0275 






Practicum, internship, field experience, co- 
op experience, or clinical assignment 


0.066* 


0.068 


0.0046 




Received prompt feedback from faculty on 
your academic performance (written or oral) 


0.055 


0.055 


0.0030 




Tutored or taught other students (paid or 
voluntary) 


0.038 


0.038 


0.0014 




Institutional contribution: Understanding 
people of other racial and ethnic 
backgrounds 


-0.074* 


-0.071 


0.0050 




Quality: Your relationships with other 
students 


0.117* 


0.115 


0.0133 




Hours per 7-day week spent working for 
pay OFF CAMPUS 


-0.012 


-0.012 


0.0002 




Talked about career plans with a faculty 
member or advisor 


-0.077* 


-0.077 


0.0059 




Had serious conversations with students 
who are very different from you in terms of 
their religious beliefs, political opinions, or 
personal values 


-0.054 


-0.054 


0.0029 




Institutional contribution: Acquiring job or 
work-related knowledge and skills 


0.066* 


0.066 


0.0043 




Institutional emphasis: Attending campus 
events and activities (special speakers, 
cultural performances, athletic events, etc. ) 


0.084* 


0.084 


0.0070 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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Appendix 



Regression Models (cont’d) 



Senior: Job Placement at Graduation 



Model Summary 

Change Statistics 

Adjusted R Std. Error of the r Square Sig. F 



Model 


R 


R Square 


Square 


Estimate 


Change 


F Change 


dfl 


df2 


Change 


1 


.514 


.264 


.256 


.394 


.264 


33.705 


1 


94 


<0.001 


2 


.734 


.539 


.484 


.328 


.275 


5.625 


9 


85 


<0.001 



1: Predictors: (constant) Employment control(Unstandardized Predicted Value) 

2: Predictors: (constant) Employment control(Unstandardized Predicted Value) 

Practicum, internship, field experience, co-op experience, or clinical assignment 
Institutional contribution: Working effectively with others 

Coursework emphasized: MAKING JUDGMENTS about the value of information, arguments, or methods, such as examining how others 
gathered and interpreted data and assessing the soundness of their conclusions 
Worked with classmates OUTSIDE OF CLASS to prepare class assignments 
Community service or volunteer work 

Number of written papers or reports of 20 PAGES OR MORE 
Institutional emphasis: Attending campus events and activities 

Work on a research project with a faculty member outside of course or program requirements 
Institutional emphasis: Providing the support you need to help you succeed academically 



Variables 


Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Reported employment at time of graduation (0 = No, 1 = Yes) 




.484* 


Control Factor 


0.410* 


0.487 


0.2375* 




Practicum, internship, field experience, co- 
op experience, or clinical assignment 


0.196* 


0.258 


0.0666* 




Institutional contribution: Working 
effectively with others 


0.256* 


0.303 


0.0916* 




Coursework emphasized: MAKING 
JUDGMENTS about the value of 
information, arguments, or methods, such as 
examining how others gathered and 
interpreted data and assessing the soundness 
of their conclusions 


-0.334* 


-0.429 


0.1840* 




Worked with classmates OUTSIDE OF 
CLASS to prepare class assignments 


0.111 


0.146 


0.0212 




Community service or volunteer work 


0.252* 


0.321 


0.1028* 




Number of written papers or reports of 20 
PAGES OR MORE 


-0.114 


-0.158 


0.0251 




Institutional emphasis: Attending campus 
events and activities 


-0.119 


-0.157 


0.0246 




Work on a research project with a faculty 
member outside of course or program 
requirements 


-0.046 


-0.064 


0.0041 




Institutional emphasis: Providing the 
support you need to help you succeed 
academically 


-0.035 


-0.044 


0.0019 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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Appendix 



Regression Models (cont’d) 



Senior: Flans For Further Education 
Model Summary 



Change Statistics 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of the 
Estimate 


R Square 
Change 


F Change 


dfl 


df2 


Sig. F 
Change 


1 


.535 


.287 


.280 


.801 


.287 


47.389 


1 


118 


<0.001 


2 


.665 


.442 


.418 


.720 


.156 


7.966 


4 


114 


<0.001 



1: Predictors: (constant) Continuing Ed control(Unstandardized Predicted Value) 

2: Predictors: (constant) Employment control(Unstandardized Predicted Value), Talked about career plans with a faculty member or advisor, 
Practicum, internship, field experience, co-op experience, or clinical assignment, Put together ideas or concepts from different courses when 
completing assignments or during class discussions. Hours per 7-day week spent working for pay OFF CAMPUS 





Std. Beta 


Partial 

Correlation 


R 2 for Partial 
Correlation 


Adjusted R 2 for 
Model 


Future Education Plans 

(1 = does not plan to, 2 = considering, 3 = applying, 4 = currently accepted 






.418* 


Control Factor 


0.462* 


0.465 


0.2163* 




Talked about career plans with a faculty 
member or advisor 


0.294* 


0.346 


0.1197* 




Practicum, internship, field experience, co- 
op experience, or clinical assignment 


-0.239* 


-0.293 


0.0860* 




Put together ideas or concepts from 
different courses when completing 
assignments or during class discussions 


0.207* 


0.264 


0.0697* 




Hours per 7-day week spent working for 
pay OFF CAMPUS 


-0.056 


-0.064 


0.0040 




* for Beta: Significant contributor to regression model (p < .05) 

* for R-square: small effect size or greater (>.03) 

Bold items were not included in benchmarks 
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